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Abstract 

The GEAR UP graduation coach intervention developed by the GEAR UP Learning Centers at 
Western Michigan University addresses the issue of academic failure of at-risk students in high 
school. This personalized early intervention strategy begins by assessing students’ unique 
circumstances, academic histories, and strengths and weaknesses in 9th grade. Coach and student 
work together, networking with parents and teachers, to establish a plan that will bolster 
academic achievement. Intensive graduation coaching identifies internal and external resources 
to ensure the student’s success and closely monitors the student’s progress. This intervention has 
been implemented successfully in two urban school settings and results indicated substantial 
benefits. This study examines a replication of this model in a rural school setting for one year 
with an incoming 9th grade class. The process involved several phases: program preparation and 
staff and student selection; implementing the coaching intervention and monitoring progress for 
informed, data-driven decision-making; and final summative evaluation. Two general questions 
were of particular research interest: Could students be identified proactively who are likely to 
benefit from this program? Did students selected as part of the coaching caseload improve their 
grades? A longitudinal, retrospective baseline study provided matched comparison groups for 
examining the effects of the intervention treatment. Results show that these graduation coached 
at-risk students as a group performed academically in core courses almost an entire letter grade 
better than would have been anticipated based on comparative student results from baseline 
studies. Well over 50% of the treatment students, all of whom were predicted to fail, actually 
succeeded well enough to be prepared to progress to the next grade level. In this instance, 
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graduation coaching appears to have significant potential in rural as well as urban schools to 
promote student retention, success, and perseverance to graduation. 
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Objectives 

The current study results from a project selected nationally and competitively in 2009 and 
again in 2010 in dual funding rounds by Research Triangle Institute (RTI) International, for 
further evaluation as a national GEAR UP (Gaining Early Awareness and Readiness for 
Undergraduate Programs) Promising Practice (Carpenter & Zeller, 2011). This study documents 
the impact of placing graduation coaches as a GEAR UP intervention in a high-need rural high 
school. Three middle school cohorts (8th graders in middle school in 2007, 2008, and 2009) and 
continuing in high school into and through 2011 served as a comparative baseline. These 
students received no intervention but were followed through 9th grade and beyond. This study 
followed a fourth cohort of 8th grade students in 2010 into high school and through 9th grade to 
evaluate the effectiveness of a graduation coaching intervention designed for at-risk students and 
tested previously in urban school district settings (Van Kannel-Ray, et al., 2008, 2009; Lacefield, 
et al., 2010). The overall goal was improvement of students’ academic performance and 
retention, particularly for students not passing core courses. 

Perspectives and Theoretical Framework 
High school student dropout is a major international problem (Oreopoulos, 2007). On 
March 1, 2010, addressing the America's Promise Alliance, President Obama emphasized the 
critical issue of high school drop-out rates in our country and stated that increasing graduation 
rates is a key goal of the administration. The U.S. Department of Education’s GEAR UP 
program addresses this problem through goals that promote graduation and post-secondary 
education. The GEAR UP program at Western Michigan University developed interventions that 
achieve these goals by helping students at risk of failure succeed academically, thereby 
increasing the likelihood of graduation and potential enrollment into post-secondary education. 
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One particularly successful intervention that has helped improve student retention in urban high 
schools is graduation coaching (Van Kannel-Ray, et al., 2009; Lacefield, et al., 2010). The 
current study looks at a similar program in a rural setting. A goal is to demonstrate that the major 
components of such programs: (1) selecting students proactively, (2) intervening successfully, 
and thereby (3) keeping students in school who might otherwise have dropped out beyond 9th 
and 10th grade; combine to form an effective model that can be replicated in other rural settings 
across America. 

Retaining students - particularly multi-cultural students - in high school and facilitating 
their graduation encompasses more than an academic approach. Our concept of case 
management practices in middle schools and graduation coaching in high schools has its roots in 
the constructivist philosophy of the 1990's and the 2000's, good reviews as well as criticism of 
which are provided by Liu and Matthews (2005) and by Hmelo-Silver, Duncan, & Chinn (2007). 
As Cobb (1996) discussed in a definitive collected work, Constructivism: Theory, perspectives 
and practice, 2nd edition , edited by C.T. Fosnot (2005), the microcosm of the school influences 
student learning within a socio-cultural context. Therefore, interventions that focus on 
understanding and working within such a framework are especially important. Graduation 
coaches help students come to grips with changing cognitive structures within a social context, 
some of which, as Lewin (1995) argued, are culturally embedded. Utilizing that socio-cultural 
contextual information, graduation coaches help students focus on co-constructing environments 
in which they understand how their personal worlds and their educational worlds intersect and 
how they can succeed both socially and academically. 

While many interventions have been proposed to mitigate this growing problem, 
graduation coaches have been shown to be multi-culturally responsive and effective in urban 



GRADUATION COACHING IN A RURAL DISTRICT SCHOOL 


6 


high schools for students who are not passing core courses and who are at-risk of dropping out of 
school (Cox, 2009). Graduation coaching provides a structural support within a building/grade 
level that buffers against the difficult transition between middle and high school. Furthermore 
coaching provides student-level support for the social and behavioral challenges that adolescents 
face as they transition to adulthood. Providing a positive learning environment within which 
students can cope with the challenges and barriers in their lives so that they can succeed 
academically (Lee, 2005) is particularly important for children who live in low-income, rural or 
urban settings. In this case, the graduation coaches, who are the structural supports, deliberately 
facilitate academic success for at-risk students. 

What we describe here as "case managers" in middle school and "graduation coaches" in 
high school is now also being widely promoted and implemented as "academic coaching" in 
community colleges, technical schools, and higher education. These post-secondary services go 
well beyond customary faculty advising and are being developed as a result of emphasis on 
"data-informed decision-making" and on innovative practices that have potential for increasing 
student retention and degree completion (issues reaching desperate proportions for many post¬ 
secondary institutions in an era of austerity and non-traditional students). Individual institutions 
(Little, 2011) and major software companies (Ellucian, 2013) are developing "retention 
software" to support academic coaching using methodologies similar to those underlying our 
own decade-long GEAR UP research and innovation activities for at-risk students. Perhaps one 
of the most comprehensive reviews of literature on factors influencing post-secondary student 
success (and likewise high school student success) is provided by a recent commissioned report 
for the National Symposium on Postsecondary Student Success (Kuh, G.D., et al., 2006). 
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Study Location, Population, and Research Questions 

Over the past decade, according to information from the State of Michigan’s Center for 
Educational Performance and Information (CEPI Michigan Student Data System, 2010), our 
partner high school in southwestern Michigan has experienced an excessive student drop-out rate 
exceeding 40 %. This has been particularly noticeable in GEAR UP project data trends early in 
9th grade and continuing through the 10th grade. Students who are promoted to the 11th grade 
typically complete high school successfully and graduate. For these reasons, an intensive 
graduation coaching intervention was planned and targeted specifically toward services for 
incoming 9th grade students who appeared at risk of future academic failure. This was expected 
to help students increase the number of core courses passed and improve marking period grades 
in core subject areas. 

The intervention was planned to take into account the unique needs of the rural student 
population, the nature of small communities, and local economic conditions. With a local 
population of approximately 6,500, the district is in a small, struggling, rural, agricultural 
community in southwest Michigan with virtually no business or industrial base. The county has a 
child poverty rate of 25 %, ranking 25th in child poverty of the 28 similar counties in Michigan. 
The district has one high school and one middle school with 76 % of all students eligible for free 
and reduced lunch and home to a significant migrant population. 

General goals of particular study interest included to what extent: 

• Can students who are likely to benefit from this program be identified proactively (based 
primarily on available longitudinal school data on academic performance and secondarily 
on data regarding attendance, behavior, and teachers’ and other stakeholders’ 
perceptions)? 

• Do students selected as part of the coaching caseload improve their grades? (Do 
academic performance patterns begin to slope away from the predicted direction toward 
failure?) 
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Intervention 

Borrowed from health care and social services professions, the graduation coach 
intervention is relatively new to the educational sector. In medical and human services arenas, 
case managers have a number of functions that include identifying client needs, developing a 
plan for helping clients, securing outside services to scaffold the interventions with the client, 
and periodically reassessing the plan (Piette, et al., 1990). GEAR UP Learning Centers personnel 
believed that the graduation coach intervention could serve similar functions in middle and high 
school settings. 

The Graduation Coaching Intervention 

This intervention began as an academic case management program in feeder middle 
schools in a participating urban school district for the express purpose of improving student 
academic achievement. The results of the data analysis for the middle school intervention 
indicated substantial success (Van Kannel-Ray, et al., 2008, 2009). However, as the GEAR UP 
student cohorts moved into high school, the middle school model was revised to incorporate 
strategies appropriate to the needs of high school students. 

The developmental needs of high school students required a more personal approach. For 
instance, high school students expect to make their own choices. Students were offered the 
option of participating in the program. Students worked with the coach to develop an individual 
Student Plan for academic success. The plan contained a series of questions that assisted the 
coach in outlining specific interventions for the student, determining who was responsible for 
each intervention, when the intervention was to take place and its duration. The plan also 
conveyed intended outcomes so that students, parents, school personnel, and project staff had the 


necessary information available to them in order to collaborate on behalf of the students 
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involved. Students demonstrated their commitment to their plan with their signature. In middle 
school, students were able to leave class for coaching when necessary. In high school, 
appointments were scheduled with students. This approach reinforced the serious nature of the 
appointment and the intervention. High school teachers appreciated fewer interruptions as 
students were able to assume responsibility for leaving the classroom to meet with the coach. 

Following two years of testing and modification, the elements of a successful high school 
graduation coaching model emerged (Van Kannel-Ray, et al., 2009). Early evaluation of these 
pilot study interventions indicated that graduation coaching is a promising practice in high-need 
urban high schools (Lacefield, et al., 2010). The intervention described here is a modification of 
the existing urban graduation coaching intervention aligned to the needs of rural schools. Rural 
schools are smaller, have fewer financial and staff resources, fewer community resources, and 
generally more transient student populations than larger urban neighbors. 

The Graduation Coaching Logic Model presented in Figure 1 reflects the graduation 
coach intervention, its context, inputs, processes, and outcomes. The model begins with a 
number of specified inputs that designate the people and the sources of information necessary to 
implement the process. Project staff from Western Michigan University, in collaboration with 
teachers, administrators, and the counseling team from the partner high school, engaged in 
continued planning, coordination, and formative evaluation activities. The experiences and 
findings from past studies in urban settings assisted the intervention team plan staffing and 
identify potential barriers to student identification and program implementation. 
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Figure 1. Program: Graduation Coaching Logic Model. 


Situation: Ninth grade students identified at-risk based on 

longitudinal data from sixth through eighth grade are 
failing core courses and not on track for graduation. 


Unique to school settings, graduation coaching is a method for uncovering the contextual 
frameworks that students attribute to their world. Building on that contextual awareness, the 
coach can help students co-construct viewpoints from which they can understand how their 
personal world and their educational world intersect (Gergen, 2001). The shared goal is to 
increase students’ academic success in school. The coach assists students to understand concepts 
being taught, complete classroom assignments, and receive credit for their work. One way of 
doing this is by providing or arranging tutoring services. It was proposed that the coach would 
meet once per week at minimum with each student; seek to develop a trusting, respectful, 
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mentoring relationship to help the student help himself or herself; communicate with parents or 
guardians; keep case records; and meet with teachers as an advocate for the student. 

Coaching Outcomes 

This intervention strives toward five key outcomes. First, the graduation coach mentors 
students toward becoming self-sufficient and academically successful. Second, the coach 
regularly helps failing students come to an understanding of the importance of completing 
schoolwork. Third, the coaching process begins to foster a culture within a school that focuses on 
learning and continued academic endeavors beyond high school. Fourth, the coach seeks to 
empower significant adults in students’ lives such as parents, grandparents, and other family 
members to advocate for their children. Fifth, teachers’ respect for the coaching process is earned 
through visible effects on coached student attitudes and work ethics. Such intervention strategies 
hold great promise for motivating and encouraging students to become responsible for their own 
learning. The first outcome was examined quantitatively in this study, while the other outcomes 
were approached qualitatively. 

During the summer of 2010, the coaching position was advertised and a graduation coach 
was hired at 50% time. This individual was an experienced high school counselor who was also 
enrolled in the doctoral program in Western Michigan University’s Department of Counseling 
Education and Counseling Psychology. Although the graduation coaching intervention has some 
overlap with counseling and mentoring programs, its primary features focus on 10 key coaching 
activities for student academic success, as do the quantitative and qualitative indicators used for 
formative and summative evaluation. In this study, the first two activities were required of the 
coach for all students and the other activities were offered to every student on the caseload. 

1. Developing/monitoring/modifying an educational development plan. 
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2. Meeting with students on a weekly basis whenever possible, with additional contact as 
needed. 

3. Tutoring or arranging for tutoring services in core subjects. 

4. Assisting teachers with developing specific interventions/activities/help for their failing 
students as well as observing classes to understand content in order to better assist 
students. 

5. Becoming a liaison between school and home for identified students to encourage 
academic improvement. 

6. Connecting students to school services such as after school tutoring, counseling, 
academic clubs, athletic activities, and organizations. 

7. Assisting students with organizational skills and study skills. 

8. Referring students and parents to outside school services to help remove barriers to 
educational success. 

9. Monitoring the student’s academic progress in all their courses at least weekly. 

10. Maintaining a student interaction record and dossier for monthly formative and 
summative activity reporting. 

Initial Contact 

The graduation coach began the process with an initial student contact. In reviewing the 
initial contact for each student the coach noted that students were open to conversation and 
assistance from the coach. Parallel to meeting with students, the coach began reviewing 
curriculum files and electronic gradebook records for each student on the case load. With the 
notes from these files, the coach then began to document grades and missing assignments for 
students. Both the notes from the curriculum files and grades and missing assignments allowed 
the coach to begin the process of the writing individual student plans. These plans allowed the 
coach and students to focus on and work through challenges, address specific needs, and 


celebrate accomplishments. 
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Periodic Contact and Review 

The process continued with a series of conversations between the students and the coach 
with discussions of what assignments had and had not been completed. The coach shared the 
tracking of assignments with the students and discussed what was necessary to complete missing 
assignments. This communication evolved over the course of the study into routine 
communication with students’ teachers and parents or guardians. These communications 
naturally flowed into discussions concerning barriers to completing assignments posed by social 
and environmental factors outside school and to behavioral, organizational, and foundational 
skills needed for success in school. The coach also arranged tutoring sessions during and after 
school and a series of student incentives including written positive communication to parents and 
guardians about the students’ academic performance. 

Data-Driven Decision Making Tools 

The graduation coaching intervention examined here depended strongly on three 
additional tools not previously employed fully in urban settings. A first key objective and 
research goal for this study was to demonstrate computer based analytical and data visualization 
programs as effective methods to proactively identify students who (a) appear to be trending 
successfully in school in terms of academic performance in core courses and subject matter areas 
such as mathematics, science, language arts, and social studies, as well as students who 
(b) appear to be at substantial risk of academic failure in these areas. 

The first tool was a “classification dashboard” report to help identify students who 
appear to be at substantial academic risk for failure, what some of those risks are, and who might 
or did benefit from specific academic support interventions at both the individual and aggregate 
level. The dashboards utilized a suite of longitudinal data analysis and data visualization tools 
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such as pivot charts built using SAS 9.2 and MS Access 2010 that drew upon readily available 
school student information system data sources. This suite allowed the program team to validate 
the methodology using retrospective studies of previous cohorts of 9th graders in the high 
school. It also served to identify a subpopulation of the current 2010-2011 cohort of 9th graders 
who appeared academically at risk. That subpopulation formed the basis for the graduation 
coaching caseload which became the treatment group in this study. Periodic updates in the form 
of individual student dashboards showing on-going academic performance trends provided 
formative feedback to the coach, school personnel, and program team at the end of each marking 
period quarter. Lacefield, et al. (2012) provides extensive discussion of this methodology and 
examples of how classification and data visualization tracking tools can facilitate educational 
decision-making at the student level. 

The second tool was real-time access to the school electronic grade book software and to 
current data concerning each student in the caseload. The coach was able to triangulate the 
information from the electronic grade-book with information gathered from teacher and parent 
contacts, past academic performance, and with information obtained during meetings with other 
parties on behalf of the student. 

As the third tool, the coach was provided with a modified version of an Excel-based 
Activity Reporting System used by the GEAR UP Learning Centers to monitor and document all 
service-related project activities (Carpenter & Zeller, 2011). This consisted of three parts: a 
Daily Log, an Activity Log, and a Case Notes Log for student-level documentation of process and 
qualitative factors. The Activity Reporting System allowed the coach to maintain electronic case 
notes and logs and to track the number, type, and duration of coaching contacts and activities. 
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This gave the coach management tools for organizing and documenting caseload activity and for 
planning appropriate student services. 

These tools were not simply evaluation add-ons. Their explicit purpose was to increase 
both the efficiency and the effectiveness of the intervention. That they also provided the basis for 
an extraordinarily comprehensive embedded system both for formative and summative, 
quantitative and qualitative evaluation was the additional benefit. 

Intervention Challenges 

The primary intervention challenges involved working with faculty and administration at 
the high school to organize support services to meet the individualized needs of students in the 
caseload. A major challenge was the development and/or identification of appropriate referral 
programs for individual students. This is a small farming community located in a large rural 
county. The school district is limited in support services that are easily accessible and convenient 
for parents. After-school and weekend tutoring programs for 9th grade students with core subject 
area content, flexible enough to meet individual student needs, were in short supply. The district 
was required to provide additional support services in high school math and science education in 
order to meet new requirements recently legislated by the state government. Fortunately, one 
very important service provided by the GEAR UP project and sustained in both middle and high 
school by the district was a comprehensive student counseling program. Counseling services 
were offered and proved valuable for several students who experienced personal difficulties 
beyond the scope of graduation coaching. 

Difficulties communicating with parents were yet another intervention challenge. 

Parents’ work schedules limited the coach’s ability to contact them during the day, and in the 
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evening. Some fa mi lies did not have telephone or internet access. The coach was able to 
overcome this problem by seeking out parents at school events such as sports or conferences. 

Procedural Plan and Research Design 

It was anticipated that students who demonstrate a consistent pattern of diminishing 
academic performance prior to 9th grade, which might lead to failure and drop-out, could be 
identified and targeted for intensive graduation coach intervention. This student group formed 
the initial caseload. A favorable outcome would be a leveling or an increase in marking period 
GPA. This study was designed to achieve the following specific objectives in the initial phase, 
beginning prior to and with the onset of the coaching intervention: 

• Recruitment: Implement, demonstrate, and verify an empirical identification/selection 
process based upon available school data (as well as teacher recommendations and other 
qualitative sources) that would be replicable in other school settings and that would 
clearly identify students who appear on a track headed toward academic failure in high 
school. Utilize this method to identify and recruit students for the coaching caseload. 

• Treatment Intervention: Implement an appropriate academic coach-based support 
system for those students identified as at-risk of academic failure that would be guided by 
real-time process monitoring, formative evaluation, and information feedback. 

Classification Methodology 

A general method for classification of student academic performance trends (Lacefield, et 
al., 2011) was utilized in this study. In a previous study, this method was applied to data 
regarding three historical cohorts totaling N=213 students from this rural school district who 
were in 8th grade in 2007, 2008, and 2009. None of these students received academic support 
services. Two groups were identified: A= 100 students who appeared to be on a clear downward 
trajectory and very much at risk of academic failure and a peer comparison group of A=173 other 
students who appeared successful and relatively well prepared for high school. Ninth grade 
students in the at-risk group continued to plummet, well beyond the GPA <1.0 point. 
Approximately 75% of the 8th grade students identified as at-risk of failure did fail in the 
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9th grade. Likewise, 94% of the students expected to pass did so. These results validate the 
classification method as an appropriate tool for identifying at-risk students. 

Research Design 

The sampling plan for this causal-comparative study (Table 1) consisted of the 2010 
cohort of 94 previously unclassified students who completed 8th grade in 2009-2010 and were 
incoming 9th grade students in 2010-2011. Based on statistical analysis of longitudinal 
performance data, 36 of these students were classified as at-risk and formed the treatment group. 
During the first quarter, three of these students left the school and one student previously not 
classified as at-risk was reclassified. During the fourth quarter, another student also previously 
not classified as at-risk was recommended for and received coaching. Thus the sample of 
students who received coaching during the intervention was 35. Grade data was not available for 
students who did not complete the 9th grade and they were not included in the final analysis, 
reducing the treatment group in the study to 32. In addition, 100 of the 273 students in the 
previous three baseline 8th grade cohorts (2007, 2008, and 2009) were classified as at-risk but 
never received coaching. These students and their peers not classified as at-risk formed the 
baseline comparison groups, demonstrating what happens without the coaching intervention. 

Table 1 

Sampling and Experimental Design 




Longitudinal Observation Period 

Current Study Period 

Target Study 

6th Grade 

7th Grade 

Sth Grade 

9th Grade 

Cohort* 

TREATMENT GROUP 

N 

MP1 

MP2 

MP3 

MP4 

MP1 

MP2 

MP3 

MP4 

MP1 

MP2 

MP3 

MP4 

Q1 

Q2 

Q3 

Q4 

2010 

Peer Comparison 

62 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

Graduation Coached 

32 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

0 

X* 

X* 

X* 

X* 

Historical Baseline Study 

















2007- 

Peer Comparison 

169 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

0 

0 

0 

2009 

At-Risk, Not Coached 

100 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

O 

0 

0 

0 

0 


' Y ear of end of 8th grade 


| * Graduation coaching interven tion 
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Treatment and Control Groups 

To illustrate how this process works, the marking period GPA in core academic courses 
was the dependent variable. What GPA means and how valid and reliable such measures are can 
be argued indefinitely. For purposes of this study, GPA is understood as a measure of the degree 
to which a student has mastered the content of core academic courses and has satisfied his or her 
teacher regarding mastery of a particular content studied within a particular period of time. 

Past research shows that academic marking period GPA dips somewhat as the students 
make the transition from middle to high school (National High School Center transition report, 
2007). Students encounter new requirements and responsibilities and pressures in high school. 
Therefore, the graduation coaching intervention began at the start of the academic year with 
students selected prior to this point. The selection process was allowed to extend through the first 
quarter of 9th grade, so that students experiencing difficulties in specific courses or with 
transition to high school could also take advantage of the graduation coaching intervention. 

Both the historical baseline and the treatment groups shared similar demographic and 
experiential characteristics. Comparative statistical analyses showed no significant group 
distribution differences for variables such as trend classification (successful or at-risk), gender, 
ethnicity, special education or economic status. The baseline and treatment groups also shared 
similar rural community and middle school experiences. As shown in Table 1, the baseline and 
treatment groups represented the two levels of the factor COHORT in the analytical model. The 
second factor is that the treatment GROUP had two levels: trending at-risk or trending 
successful, representing the students who were coached or would have been eligible for coaching 
and their peer comparison groups. The third factor is TIME, for data gathered repeatedly for each 
student over 4 full school years (12 marking periods in 6th through 8th grades and 4 quarters in 
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9th grade). GROUP is fully crossed with COHORT and both are nested with TIME in the 
longitudinal research design. The dependent variable of primary interest is grading period GPA. 
GPA is the average of grades in core academic coursework posted at the end of each grading 
period from 6th through 9th grade. 

Data Collection 

The intervention phase of the study began in November 2010, when the graduation coach 
actively assumed his role within the high school. This phase continued through the end of the 
academic year, in early June 2011, and for a debriefing period following that point. The 
academic year covered 173 school days. The graduation coach was employed as a regular, half¬ 
time staff member by the school district and was provided with an office in the high school with 
access to the electronic grade book and confidential student records. The coach was supervised 
by the project director, the evaluation team, and the GEAR UP site staff. Additional oversight 
was provided by the university's Human Subjects Institutional Review Board through approved 
research protocols for GEAR UP Learning Centers projects as well as this study. 

During this phase, three types of evaluation data (process, qualitative, and quantitative) 
were collected and used continually to guide the intervention and the coach’s work. Process 
evaluation data was gathered using two of the three components of the Activity Reporting 
System: Daily Logs and Activity Logs. The third component of the Activity Reporting System: 
Case Notes was used to collect qualitative data. Quantitative academic performance data from 
the school’s student information system was gathered at quarterly grading period points for 
baseline and current study students. In addition, the coach had access to the school’s electronic 
gradebook to monitor student progress between grading periods. 
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Process Evaluation Methods 

Activity Reporting System data provided information for on-going management and 
accountability. It also was essential for insuring the fidelity of the intervention and, hence, the 
validity of study assessments. Activity Reporting System components each included data 
validation controls to maintain the accuracy and integrity of the information collected. The first 
component, the Daily Log, served both as a system for recording student service information and 
as a relational database for real-time information to help the user manage on-going activities and 
collect data for program monitoring and evaluation. The data set produced by the Daily Log 
contained: 

• Date of service 

• Student name 

• Student identification number 

• Planning time in minutes for the student, parent, and teacher 

• Contact time in minutes for the student, parent, and teacher 

• Caseload status of the student 

• Total number of credit hours for the student 

• Type of intervention service for the session 

The second component, the Activities Log, was used to collect more general intervention 
and school-oriented activities. The data set produced the Activities Log contained: 

• Date of the activity 

• Type of activity 

• Intervention category 

• Number of participants 

• Type of participants 
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• Time in minutes 

Qualitative Evaluation Methods 

The third component, Case Notes, was used to identify individual student services, 
provide information for on-going assessment of academic progress, collect formative evaluation 
data, and document and report on the progress of each individual coaching session. The 
information produced by the Case Notes included: 

• The purpose of the session 

• The discussion that took place and why 

• The results of the session 

• The necessary follow-up steps with school personnel and/or parents 

In addition, an individualized Student Plan was developed at the beginning of the project. 
The plan contained the following questions to assist the coach in outlining specific interventions 
for the student, determining who was responsible for each intervention, when the intervention 
was to take place, and what would be its expected duration: 

• What are the specific needs of the student? List and explain each need (i.e., behavioral, 
academic, attendance, etc.). 

• Who are the individuals who will address these needs? List and identify who these 
individuals are (i.e. parents, counselors, administrators, teachers, graduation coach etc.). 

• How will the individuals in the above list address the needs of the student? 

• If necessary, how often will the student meet with these individuals? 

• How will performance of the student be tracked or monitored by these individuals (i.e., if 
seen by a counselor, how will improvement be noted)? 

Both formative feedback and summative understanding were goals for quantitative and 
qualitative evaluation of the program to determine the effectiveness of graduation coaching in a 
rural high school (Zeller & Carpenter, 2011). The instruments described above provided valuable 



GRADUATION COACHING IN A RURAL DISTRICT SCHOOL 


22 


information regarding the impact of the services on students and these were modified and 
extended to be useful to the coach for record-keeping, to identify student barriers, and for 
decision-making about provision of specific support services. In addition, the project team held 
monthly meetings with the coach which helped to identify successes and barriers and to fine-tune 
the coaching process. 

Quantitative Evaluation Methods 

The causal-comparative research design employed by this study followed individual 
students within each group in each cohort from 6th grade into and through 9th grade. The 
primary data source was student information system records. The primary longitudinal 
quantitative data stream consisted of marking period/quarter academic performance in core 
subjects. Benchmark points for data collection, analysis, and information feedback included the 
end of 8th grade and the prior academic year as well as each quarter of the subsequent 9th grade 
academic year during which coaching took place. 

One property of longitudinal research designs is that individual subjects and intervention 
groups can serve as their own controls. However, this study is greatly strengthened by the 
availability and use of data from prior cohorts of students who had very similar educational 
experiences in this same school district but never received personalized case management or 
graduation coaching services. The prior cohorts served as historical baseline studies for 
comparison against the outcomes of this study. 

Analysis and Results 

The Activity Reporting System 

The Activity Reporting System provided detailed information regarding the 
implementation process and the interactions between the graduation coach and the students in the 
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caseload, school personnel, counseling personnel, parents, and outside entities such as 
community health social services organizations. For example, over the course of a year, this 
system enabled us to document definitely that 79% of the coach's total available time (1029 
hours) was spent on intervention. The balance of his time, 21%, was spent in collaborating with 
school personnel and program staff and coordinating the team effort. 

Table 2 summarizes the data collected by the Daily Log for the coaching intervention 
services (as defined in Table 3, an excerpt from the Data Dictionary portion of that log). The 
graduation coach recorded data for three categories of service: Consultation, Student Planning, 
and Academic Tracking and Support. Each of the three categories was subdivided by time, 
planning time, and contact time. Contact time is defined as any communication with or on 
behalf of students. This time could include face-to-face meetings, telephone conversations, 
e-mail or text messaging, and mailings. Planning time is the time spent in preparation for contact 
with students. 

As shown in Table 2, the graduation coach spent approximately 34 hours on 
Consultation, 147 hours on the Student Plan and 116 hours on Academic Tracking and Support 
for the cohort. The average time spent on behalf of students was approximately 1 hour on 
Consultation, 4 hours on the Student Plan, and 4 hours on Academic Tracking and Support. 
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Table 2 

Daily Log Table of Graduation Coach Involvement with Students 


Number of Students in Cohort: 


35 


Consultation 


Planning 

Contact 

Total 

Average Time 

Student 

11.67 

9.75 

21.42 

0.61 

Parent 

1.17 

3.47 

4.63 

0.13 

Teacher 

1.25 

3.13 

4.38 

0.13 

Administrator 

- 

1.55 

1.55 

0.04 

Counselor 

0.33 

1.30 

1.63 

0.05 

Total Hours 

14.42 

19.20 

33.62 

0.96 


Student Plan 


Planning 

Contact 

Total 

Average Time 

Student 

32.45 

98.25 

130.70 

3.73 

Parent 

0.27 

1.13 

1.40 

0.04 

Teacher 

0.22 

10.67 

10.88 

0.31 

Administrator 

0.17 

2.33 

2.50 

0.07 

Counselor 

0.25 

1.50 

1.75 

0.05 

Total Hours 

33.35 

113.88 

147.23 

4.21 


Academic Tracking 


Planning 

Contact 

Total 

Average Time 

Student 

67.82 

27.42 

95.23 

2.72 

Parent 

0.42 

1.75 

2.17 

0.06 

Teacher 

3.70 

7.48 

11.18 

0.32 

Administrator 

0.97 

3.52 

4.48 

0.13 

Counselor 

0.08 

2.55 

2.63 

0.08 

Total Hours 

72.98 

42.72 

115.70 

3.31 


Note: 

Average time is the amount of time in hours on behalf of the student 
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Table 3 

Daily Log Data Dictionary 


Services to Students 


Consultation: 

Any type of communication with students, in person, by telephone, e-mail or text; 
or on behalf of the students with parents, teachers, administrators, counselors. 

Assist students with issues such as home problems, conflicts with other students or 
school staff. 

Contact or provide recommendations for resources, mentoring, tutoring, after-school 
programs health agencies, social services, etc. 

Include planning time, such as review of individual Student Plan, electronic grade book, 
teacher, parent, and counselor comments, coaching history, courses needed to graduate, 
etc. 

Student Plan: 

The formal plan developed with the student to determine the necessary course of action 
for academic success. Review the progress on the tasks associated with the Student 
Plan to determine adjustments as needed. 

Include planning time, such as review of individual Student Plan, electronic grade book, 
teacher, parent, and counselor comments, coaching history, courses needed to graduate, 
etc. 

Academic Tracking and Support: 

Monitor academic progress and work with students to specifically support scholastic 
achievement geared toward graduation. 

Assist students in organizing time, managing homework assignments, prioritizing 
workloads, discussions on assignments, test preparation, reviewing test scores, etc. 

Include planning time, such as review of individual Student Plan, electronic grade book, 
teacher, parent, and counselor comments, coaching history, courses needed to graduate, 
etc. 




GRADUATION COACHING IN A RURAL DISTRICT SCHOOL 


26 


Table 4 summarizes Activity Log data. This shows that the coach was involved in 
meetings and other activities on at least 63 different occasions with 182 participants. Contacts 
are defined as any communication with or on behalf of students. This time could include face-to- 
face meetings, telephone conversations, e-mail or text messaging, and mailings. Table 4 shows 
the number of contacts with participants, the administrators, teachers, counseling staff, and the 
evaluation team. The participant contacts focused on school relations, monthly meetings with 
the evaluation team and other project-related administrative duties. 

The process and contact distributions correspond to the design of the graduation coaching 
intervention and are key fidelity features for assessing verisimilitude of the graduation coaching 
process. The Activity Reporting System accounted for well over 83% of the coach’s time 
available to the program. It should be noted that the additional time received by each student 
from support services personnel to whom students were referred is not included here but 
represents a truly significant multiplier effect to be explored in other studies. 

Table 4 

Other Activities Table of Graduation Coach Involvement with Other Persons and 

Activities 


Summary 

Number of 

Number of 

Total Time 

Intervention Specialists 

Contacts 

Participants 

Hours 

Consulting with Administrator(s) 

28 

52 

27 

Consulting with GEAR UP Counselor(s) 

9 

17 

8 

Consulting with School Counselor(s) 

1 

1 

0 

Consulting with Teacher(s) 

15 

51 

9 

School Relations 

3 

55 

5 

Project Administrative Duties 

7 

6 

12 

Total 

63 

182 

60 
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Qualitative Data Analysis, Findings and Conclusions: 

A variety of instruments were developed and used to gather qualitative data preceding 
and throughout the implementation process. Table 5 summarizes qualitative data gathered from 
the Case Notes. These notes were written by the graduation coach after each encounter with the 
student, teacher, administrator, parent, or other individual pertinent to the student. 

Table 5 

Coaching Activity Tallies from Graduation Coach Case Logs 


Graduation Coach Activities 

Contacts 

Initial student contact 

35 

Prepare student plan 

70 


Discussion with administrators 

37 

Discussion with counselors 

35 

Discussion with evaluation team 

6 

Discussion with parents 

24 

Discussion with students 

208 

Discussion with social workers 

3 

Discussion with teachers 

49 


Counsel students 

32 

Counseling referrals 

5 


Review grades with students 

330 

Homework assistance 

96 

Organizational assistance 

25 

Testing assistance 

18 


The process of working on a case-to-case basis with students included much 
communication with and support from a variety of other individuals who had significant roles in 
supporting the student’s academic performance. The number of student contacts, discussions, 
assistance with homework, testing and reviewing grades, far exceeded other contacts. Contacts 
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are defined as any communication with or on behalf of students. This time could include face-to- 
face meetings, telephone conversations, e-mail or text messaging, and mailings. The review of 
grades and missing assignments (630 events) indicates that the coach and students were seriously 
involved in attempting to remedy issues and raise grades in deficient subject areas. The coach 
also assisted students with preparation for tests and quizzes. The Case Notes Logs indicated that 
many of these students had problems with organizing for and taking tests. From the case notes, it 
appeared that the students recognized the benefit of organizational study and test-taking skills. 

Case Notes captured a number of events that occurred during coach and student meetings. 
These notes offered rich details of what took place during one-on-one interactions. Different than 
the Daily Log that captured a meeting with a student, parent, or teacher, Case Notes provided 
detailed descriptions of what took place during these meetings. So one meeting recorded on the 
Daily Log might result in two or more events that took place during that meeting. The coach 
could, for example, meet with a student and mentor the student, then review grades, and also 
assist in setting up tutoring sessions or other sources for outside assistance. 

Based on student progress, the coach determined how frequently the student needed to 
meet during the academic year. Likewise, at regular grading period intervals, feedback in the 
form of individual student classification dashboards helped the coach identify and target specific 
areas of academic performance that required additional intervention efforts. Collaboration, 
communication, and cooperation are the hallmarks of any successful school related endeavor, 
and these were exemplified through this study. Table 5 reflects the number of discussions that 
the coach had with school personnel, students, parents, and others. The students receiving the 
coaching assistance benefitted from the many conversations that focused on their wellbeing. 
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Quantitative Data Analysis: 

Three specific research questions examined by this study were: 

* Can the increasing gap in academic performance between students at-risk and their 
successful peers be slowed or reversed by appropriate intensive academic coaching? 

* Did at-risk students who were coached do better in terms of average Core GPA in 9th 
grade than similarly at-risk students who were not coached? 

* Was there a difference in the academic performance patterns of the students who were 
coached and the students who were not coached in GROUPS over TIME? Was the effect 
in the direction expected (to close the gap and improve at-risk student performance) and 
did this occur at the time expected (the 9th grade)? 

The statistical analysis showed that in this study, the average performance of the peer 
comparison student group (i.e., those trending successful) was almost flat over time at a 
performance level of approximately a GPA of 3.0. Prior to 9th grade, the average performance of 
the at-risk student group had been declining and was approximately a GPA of 1.5 at end of 8th 
grade. Almost all of the students shown in the at-risk group were predicted to fail in 9th grade. 
The actual outcome, however, was much different. The at-risk students who received graduation 
coaching in 9th grade fared much better than did similar students in the baseline study. In fact, 
for the students who received graduation coaching, the expected and predicted decline flattened 
and the average performance at end of 9th grade is close to a GPA of 2.0. This was much better 
that the performance of similarly at-risk students in the baseline study where nearly 75% failed 
in 9th grade. 

The first research question investigated whether the increasing gap in academic 
performance between students at-risk and their successful peers can be slowed or reversed by 
appropriate intensive academic coaching. Figure 2 is a simple means plot that reveals the 
observed effect of graduation coaching in the 9th grade. The dashed line in this figure shows the 
progression of 100 baseline study students who were identified at-risk but did not receive this 
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type of intervention. The solid line presents the 32 students in the analysis identified as at-risk 
who received coaching. The effect of the treatment compared to the baseline is seen during the 
9th grade. The COHORT by TIME interaction effect in the statistical model captures the central 
hypothesis of this study and was statistically significant (Wald jf df=is = 150.33, p < .0001). As a 
group, students classified at-risk who receive graduation coaching appear to earn significantly 
better grade point averages in core coursework by end of 9th grade than do similar students who 
do not receive coaching services. 


2010 Cohort with At-Risk Students Receiving Graduation Coaching in 9th Grade 
Compared with 2007, 2008, 2009 Baseline Un-Coached At-Risk Students 

Legend Baseline At-R:s«. Students Coached At-Risk Students 



Academe Years / Grading Periods Timeline 

Figure 2. Means plot and statistical results comparing academic performance 


trajectories. 
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Two additional specific planned contrasts provide more specific analytical tests. One of 
these answered research question two and looked directly at the COHORT by TIME interaction 
effect beginning at the start of 9th grade between middle school and high school. This also was 
highly significant (Wald ^ df=i = 22.84, p < .0001) and showed (1) that the trends for at-risk 
students who were coached and those who were not coached go in different directions and (2) 
that this effect begins at the onset of coaching. The other planned contrast answered research 
question three by examining the average post-intervention academic performance difference 
between current and baseline at-risk students during the 9th grade. Coached students performed 
almost an entire letter grade better than expected based on comparative student results from 
baseline studies (Wald x 2 df=i = 47.31, p < .0001). Well over 50% of the treatment students, all of 
whom were predicted to fail, actually succeeded well enough to progress to the next grade level. 

Summary and Conclusions 

In summary, the goals posed in the perspectives section were answered positively by this 
study. A reliable and longitudinal data analysis and classification procedure was demonstrated. 
Moreover, students who were identified as at-risk using this procedure did benefit academically 
as a group from graduation coaching. In this study, with a well-designed and implemented 
graduation coaching program, students selected as part of the coaching caseload did improve 
their grades, by almost an entire letter grade, in comparison to similar but not coached baseline 
student groups. Finally, this study demonstrated that graduation coaching can be as successful, if 
not more so, in a rural setting compared to the published results obtained with similar models in 
urban settings. 

The Activity Reporting System and other checks and balances insured that the graduation 
coaching program was implemented with high fidelity at the high school. With the exception of a 
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few minor challenges, the program continued throughout the school year and reached the 
students on the targeted caseload. The coach used the instruments developed for data collection 
and monitoring for purposes of intervention planning and implementation. Student records were 
reviewed; plans were written; and students, school personnel, parents and others were contacted 
on behalf of the student. Based on the fidelity and process measures, there is compelling 
evidence to believe the academic gains observed were a result of the graduation coaching 
program. 

Several of the students in the caseload continued to perform poorly in spite of the 
intervention. This appeared to have been a result of personal crises well beyond the scope and 
capability of graduation coaching support services. The coach continued to assist these students, 
although minimally, and in almost every case referred them to counseling or to social work 
programs. 

Replicability 

This intervention extends a similar GEAR UP Learning Centers intervention designed for 
and implemented in urban middle and high schools through the use of data-driven classification, 
visualization, and monitoring tools. Graduation coaching has been conducted in other schools 
around the country, for instance in Georgia (Cox, 2009). The clearly defined and executed 
graduation coaching program utilized here includes methodology and instruments for 
documentation of process and findings. For example, the Activity Reporting System proved 
highly effective as a tool for focusing on intervention strengths and weaknesses and improving 
program administration. The data visualization and monitoring system produced changes in the 
ways in which schools assess, intervene, and support students. Coaching is a resource and staff 
intensive process but the data visualization and student identification tools discussed here are 
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readily scalable to larger schools and school districts for caseload selection, diagnostic coaching 
and counseling, and comparative evaluation of intervention effects. 

Lessons Learned 

While the graduation coaching interventions and evaluation tools have evolved over 
several years, there remains room for small and large improvements. For example, making time 
within the coach’s schedules for home visitations whenever necessary or possible would be 
especially beneficial for those parents who do not have telephones. More generally, involvement 
and cooperation of school personnel, outside agencies, tutoring programs in the school, etc., are 
crucial to program success. Data collected throughout this study showed that the number of 
student contacts proposed was unrealistic. Activities such as test preparation, testing weeks, 
detention times, etc., have to be factored into the intervention plan. Downtime during holidays 
has to be considered too. The team also learned that graduation coaching may not be enough to 
help students overcome academic problems, because of the complexity of personal and 
behavioral problems. 

Graduation coaching is a resource intensive service provided to students for their benefit. 
Without cooperation from the students and a desire to do better in school, the intervention will be 
less than fully effective. The intervention is strengthened with parental involvement and support. 
Furthermore, especially in larger schools, there are usually many more students identified at-risk 
than there are resources for services like graduation coaching. With limited resources, the 
selection process should be well-defined and reviewed on a regular basis to ensure equity within 
the program. Using the caseload identification tools provided by this study, school personnel can 
apply professional judgment to determine which students are most likely to benefit from this type 
of intervention and those who should be referred to other programs or retained on an academic 



GRADUATION COACHING IN A RURAL DISTRICT SCHOOL 


34 


watch list. It should be explicit to all concerned that coaching may not be the right support 
service for all students. Students who are unmotivated or otherwise not responding to coaching 
with a reasonable period of time, say, two quarters, may be removed from the caseload and 
referred to a program that would be more beneficial. 

The evaluation of the coaching program provided information to the schools that 
identified categories of at-risk students and compared them to their more successful peers. 
Selected students were assigned to the coach caseload and received services designed to support 
academic achievement that were documented through a variety of methods. Students, teachers, 
parents, and administrators were apprised of all details regarding student performance. 
Communication and collaboration between all parties involved was crucial to improving the 
academic environment for students at-risk of failing, and formative evaluation and data-driven 
decision making were key to improvement of programs and services in this study. 
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